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INTRODUCTION 


Many  systems  of  programmed  instruction  involving 
computer  interaction  with  students  have  been  developed 
for  training  intellectual  skills.  The  skills  involved 
in  machinery  operation  are  not  purely  intellectual,  but 
rather  psychomotor.  These  skills  include  the  coordination 
of  relatively  independent  muscle  systems,  precision  in 
manipulative  control,  translation  of  visual  input  into 
motor  output,  and  the  continuous  integration  of  sensory 
information  concerning  machine  functions  into  motor  output. 
These  psychomotor  and  tracking  skills  are  quite  different 
from  the  intellectual  material  that  is  usually  taught  by 
CAI  techniques. 

Examination  of  the  literature  on  CAI  indicates  that 
when  it  is  incorporated  in  simulator  training  of  psychomotor 
skills,  it  is  generally  utilized  as  a means  of  controlling 
an  iterative  training  sequence.  CAI  is  generally  not 
utilized  to  give  subjects  immediate  feedback  of  psychomotor 
output  parameters.  When  a subject's  performance  is  compared 
to  an  established  criterion  or  model,  it  is  done  to 
determine  if  the  subject  has  satisfactorily  completed  the 
training  program  (Rigney,  Morrison,  Williams,  and  Towne , 
1973;  Rigney,  Towne,  and  Morrison,  J975;  Feurzeig,  Cohen- 
Lukas,  and  Schiff,  1975;  Faconti,  Mortimer,  and  Simpson, 
1970;  Charles,  Johnsons,  and  Swink,  1973;  and  Caro,  1973). 


This  information  is  not  often  used  to  indicate  the  "goodness 
of  fit"  of  a student  performance  model  relative  to  an  ideal 
model.  Feedback  of  information  indicating  the  relationship 
of  a student's  performance  to  a control  model  can  serve 
to  improve  the  subject's  performance.  Holding  (1965) 
sites  the  beneficial  use  of  such  performance  feedback 
for  training  air  to  air  gunnery  skills  as  well  as  a variety 
of  other  psychomotor  skills. 

Kelley  (1968)  discussed  the  effectiveness  of  augmented 
displays  and  controls,  such  as  quickened  and  predictor 
displays,  when  they  are  used  in  operational  environments. 
Such  displays  and  controls  allowed  subjects  to  predict  how 
their  control  actions  approximated  those  necessary  to 
achieve  the  system  goals.  Hence,  the  subjects  were  able 
to  compare  their  performance  models  against  a control  or 
ideal  model.  Real-time  feedback  methods  are  limited  in 
their  application  in  operational  systems.  There  are  many 
situations  where  such  displays  and  controls  cannot  be  used 
because  extraneous  environmental  factors  are  too  numerous. 

In  a simulator  the  extraneous  factors  are  controlled, 
which  makes  it  possible  to  establish  a control  model  for 
virtually  an  unlimited  number  of  variations  in  the 
simulator's  operation.  Augmented  displays  and  controls 
much  like  those  suggested  by  Kelley  (1968)  can  effectively 
be  developed  for  training  on  virtually  any  motor  performance 
task.  Implementing  such  displays  should  result  in  improved 
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performance  on  the  simulator,  but  may  also  reduce  the 
transfer  of  this  training  to  operational  situations  where 
the  special  displays  and  controls  are  replaced  with 
conventional  operating  equipment. 

The  advantage  of  using  such  displays  in  simulation 
training  lies  in  the  immediate  matching  of  the  motor  actions 
of  the  subject's  operational  models  with  the  control  model. 
The  problem  with  using  such  real-time  feedback  in  training 
was  considered  in  an  outline  by  Holding  (1965)  of  training 
considerations  related  to  handling  knowledge  of  results. 

Holding  (1965)  first  established  a distinction  between 
intrinsic  and  artificial  knowledge  of  results.  Artificial 
knowledge  was  defined  as  any  score,  display,  or  message 
that  was  used  in  training,  but  did  not  appear  in  the 
operating  environment  of  the  device  whose  function  was 
being  learned.  Ultimately,  each  trainee  must  learn  to 
perform  the  task  under  actual  operating  conditions;  that 
is  with  intrinsic  knowledge  of  results  only,  if  the  training 
skills  are  to  be  generalizable . A transition  from  artificial 
to  intrinsic  cues  should  be  provided  as  part  of  any 
training  routine  in  which  artificial  knowledge  is  used. 

Holding  (1965)  made  a further  distinction  between 
concurrent  and  terminal  feedback  of  artificial  cues,  and 
considered  terminal  feedback  to  be  superior  on  a number 
of  grounds.  Concurrent  feedback  tended  to  inhibit  the 
transition  to  intrinsic  cue  monitoring,  since  the  student's 
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attention  was  largely  devoted  to  the  concurrent  display. 
However,  the  case  was  not  simple.  Concurrent  feedback 
such  as  that  in  augmented  displays  may  offer  the  best 
possibility  for  initially  guiding  the  student's  motor 
output  into  an  approximation  of  the  desired  coordinated 
output  pattern,  and  can  be  considered  to  be  a means  of 
securing  a forced  response  rather  than  a channel  for  the 
communication  of  information.  Furthermore,  when  concurrent 
feedback  is  intermittent,  it  takes  on  many  of  the  character- 
istics of  terminal  feedback. 

In  this  study,  the  performance  of  subjects  on  a 
psychomotor  task  was  examined  after  they  had  received 
simulator  training  using  intrinsic  feedback  only;  intrinsic 
plus  terminal  artificial  feedback;  or  intrinsic  feedback 
plus  a combination  of  terminal  and  concurrent  artificial 
feedback. 


i 
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METHODS 


SUBJECTS 

Subjects  for  this  study  were  sixty  paid  volunteers 
• from  the  student  population  at  the  University  of  South 

Dakota  at  Vermillion.  All  subjects  were  screened  for  20/30 
corrected  vision,  and  had  driven  manual  transmission  cars. 

Because  of  the  difficulty  of  evaluating  training 
procedures  with  a repeated  measures  design,  we  used 
independent  and  non-overlapping  experimental  and  control 
groups.  Subjects  were  assigned  to  one  group  or  another  in 
the  order  in  which  they  volunteered  for  the  study.  T.>ble  1 
gives  a description  of  the  different  groups. 

The  groups  did  not  differ  significantly  in  age  or 
stated  experience  in  handling  manual  transmission  vehicles. 
This  equality  of  stated  experience  applied  to  the  female 
control  group  as  well  as  to  the  three  male  groups.  Since 
[ there  is  a fair  exposure  to  machinery-handling  for  college- 

i age  females  in  the  rural  South  Dakota  environment,  the 

r female  subject  pool  may  have  contained  a biased  sample  of 

i 

individual  volunteers,  weighted  in  favor  of  persons  with 
; such  experience  when  compared  to  the  national  average, 

t All  subjects  were  given  identical  briefings  and 

^ instructions  (see  Appendix  A),  and  all  were  told  they  were 

participating  in  a computer  teaching  experiment.  The  experi 
menters  were  aware  of  the  subjects'  status  as  being  either 
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Table  1 

Subject  Ages  Within  Groups 


Sample 

Size 

Age 

Range 

Mean 

Age 

Male  Control  Group 

15 

19-27 

20.8 

Female  Control  Group 

15 

18-28 

20.4 

Terminal  Feedback  Group 

15 

19-29 

21.0 

Graphic  Feedback  Group 

15 

18-24 

19.7 

experimental  or  control.  However,  there  was  no  indication 
that  any  contamination  of  results  followed  from  this  lack 
of  a double  blind  procedure. 

APPARATUS 

A heavy  truck  transmission  and  gear  shifting  mechanism 
was  used  as  a test  bed  for  different  formats  of  computer- 
student  interaction.  This  task  was  chosen  because  of  the 
ease  with  which  a wide  range  of  different  computer  instruction 
techniques  could  be  associated  with  it,  and  because  it  was 
generally  representative  of  a large  family  of  man-machine 
interactions  involving  foot  and  hand  control  of  rotating 
machinery. 

The  training  apparatus  consisted  of  an  actual  transmission 
unit,  clutch  assembly,  and  throttle  control.  The  transmission 
was  driven  by  a variable  speed  electric  motor  controlled 
by  the  throttle  pedal  through  a pedal  operated  clutch.  The 
student's  right  hand  operated  a direct-link  shift  lever 
through  a six  slot  plus  neutral  pattern.  Nominal  engine 
RPM  values  and  MPH  road  speeds  were  displayed  to  the 
student  on  a video  screen  using  horizontal  bar- type  scales. 

No  attempt  was  made  to  simulate  the  larger  visual  input 
of  an  actual  operating  environment.  The  same  RPM  and  MPH 
displays  were  used  whenever  intrinsic  feedback  was  utilized. 


The  information  monitored  by  the  computer  included 
input  and  output  shaft  rotation  rates  nominally  equivalent 
to  RPM  and  MPH,  respectively.  The  position  of  the  throttle 


and  clutch  pedals  and  the  gear  shift  lever  was  detected 
by  potentiometers  and  magnetic  reed  switches  and  fed  into 


i the  computer  to  be  compared  with  a stored  model  of  ideal 

f 

performance.  Each  of  the  five  analog  parameters  being 
monitored  was  digitally  sampled  at  one-hundred  samples/ 
second,  but  only  ten  samples/second  were  actually  stored 
in  the  computer  memory  for  further  reference.  The 
simulator  was  controlled  by  a DEC  PCP8-E  computer  system, 
with  32K  of  12  bit  core  memory.  For  a full  description  of 
the  computer  system  and  description  of  the  minimum  system 
required  to  operate  the  simulator  see  Appendix  F. 

TASK  AND  TEST  CONDITIONS 

The  performance  required  of  the  student  was  to  reach 
an  indicated  55  MPH  road  speed  in  exactly  22  seconds  with 
the  greatest  possible  degree  of  "smoothness"  while  shifting 
through  all  the  gears.  The  passage  of  22  seconds  was 
indicated  by  a buzzer,  and  smoothness  was  defined  in  terms 
of  maintaining  equal  increments  of  acceleration  over  time 
during  the  shifting  sequences. 

f CONTROL  CONDITION 

The  control  condition  for  learning  this  task  without 
computer  mediated  feedback  involved  reading  a manual, 
taking  a quiz  on  the  contents  of  the  manual,  further  study 
of  the  manual  until  a criterion  quiz  scor-c.  was  reached, 
and  a thirty  minute  practice  session  on  the  device  involving 

: 

L 
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twenty  4- shift  sequences.  Under  the  control  condition, 
subjects  received  only  RPM  and  speed  displays  (intrinsic 
feedback)  and  received  no  performance  feedback  after  each 
trial  (terminal  feedback) . Since  the  computer  display  unit 
was  used  to  present  MPH  and  RPM  data  in  this  "control" 
condition  (see  Figure  1),  and  since  the  computer  was  also 
used  to  demonstrate  the  desired  twenty-two  second  execution 
interval,  students  in  the  control  group  could  believably 
be  told  to  consider  themselves  part  of  a "computer  education" 
experiment,  even  though  no  actual  feedback  of  individual 
performance  parameters  was  involved.  All  feedback  in  the 
control  condition  was  intrinsic. 

EXPERIMENTAL  CONDITIONS 

Two  experimental  groups  were  run,  each  receiving  a 
different  form  of  performance  feedback.  Each  experimental 
condition  was  similar  to  the  control  condition  in  that  the 
participants  first  studied  the  instruction  manual  until  a 
criterion  test  score  was  reached.  The  same  demand  scheduling 
of  twenty  4-shift  sequences  were  used. 

The  experimental  treatment  differed  from  the  control 
treatment  in  that  different  types  of  supplementary  feedback 
were  provided  for  each  of  the  two  experimental  groups  to 
supplement  or  replace  the  intrinsic  feedback  available  in 
the  control  groups.  Descriptions  of  the  two  treatments 
follow. 
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Figure  1 

Instrumentation  Displayed  During  All 
Intrinsic  Real-Time  Feedback  Conditions 
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Numerical,  Terminal  Feedback  (Terminal  Group) 

Two  numerical  scores  were  calculated.  A "smoothness" 
score  was  based  on  a sum-of-squares  deviation  of  actual 
acceleration  from  the  desired  acceleration  profile.  This 
smoothness  of  acceleration  score  was  scaled  in  arbitrary 
units  from  0 to  100,  zero  representing  a perfect  score. 

A time-off-target  score  was  based  on  the  elapsed  time 
between  the  start  of  the  run  and  the  first  point  at  which 
a 55  MPH  indicated  road  speed  was  reached,  minus  the  22 
second  target  interval.  This  value  was  scaled  in  seconds 
and  tenths.  These  primary  performance  scores  were  displayed 
to  the  student  at  the  end  of  each  single  trial.  A composite 
score  and  an  historic  record  of  scores  for  earlier  trials 
was  also  displayed. 

Figure  2 shows  an  example  of  the  display  seen  by  a 
subject  on  the  eighth  of  twenty  trials.  These  subjects 
received  intrinsic  feedback  only  during  the  trial  itself. 

Using  the  performance  score  feedback,  the  subject  could  see 
at  once  the  relative  success  of  particular  performance 
attempts,  and  could  immediately  see  the  effect  of  any 
modification  or  variation  of  their  performance  tactics. 

The  subject  could  also  immediately  see  if  progress  was 
being  made  in  both  a short  term  and  a long  term  sense. 

Most  importantly,  if  a single  successful  run  was  made,  the 
subject  was  aware  of  this  at  once  while  the  proprioceptive 
sensations  of  the  performance  were  still  reasonably  immediate, 
accessible,  and  reproducible. 


11 


I 


YOUR  SCORE  ON  RUN  8 WAS  2A 
BEnER  THAN  7 BY  A POINTS 


SMOOTHNESS  SCORE:  18 


TIME  SCORE:  +1.52 


RUNS  1 2 3 A 5 6 7 8 9 10  11  12  13  lA  15 


16 


100 


Figure  2 

Terminal  Feedback  Condition- -Feedback 
on  Eighth  Trial. 
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Analytic,  Graphic  Feedback  (Graphic  Group) 

Students  trained  by  this  method  received  feedback  in 
real-time,  concurrent  with  the  actual  gear-shifting 
performance.  At  the  onset  of  each  training  trial,  the 
student  was  presented  with  a display  showing  the  plot 
of  road  speed  against  time  as  generated  by  an  ideal  performanc 
This  display  constituted  a "track"  to  be  followed  during 
the  trial.  The  student's  task  was  to  match  his  performance 
with  this  command  model.  The  student's  actual  performance 
was  represented  on  the  display  as  a dotted  line  evolving 
with  time  and  superimposed  on  the  command  track.  An 
example  of  such  a display  is  shown  in  Figure  3. 

Using  this  type  of  feedback,  the  student  saw  at  once 
when  a given  shift  execution  diverged  from  the  desired 
acceleration  pattern,  and  could  time  his  subsequent  clutch 
and  throttle  operations  accordingly.  In  order  to  effect 
a transition  from  the  concurrent  artificial  training  sessions 
to  the  intrinsic  operation  of  the  simulator,  the  following 
training  sequence  was  utilized:  For  the  first  eight 
trials,  the  graphic  feedback  was  displayed  both  concurrently 
and  terminally;  for  the  second  eight  trials,  the  real-time 
feedback  was  all  intrinsic  and  the  graphic  display  was 
shown  only  as  terminal  feedback. 

In  order  to  test  the  transfer  of  training  from  feedback 
to  non-feedback  conditions,  the  final  four  trials  were 
conducted  for  both  experimental  groups  under  intrinsic 
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feedback  alone.  Comparisons  between  control  and  experimental 
groups  were  based  on  their  scores  on  these  final  four  of 
twenty  trials.  Prior  to  these  four  trials,  both  experimental 
and  control  groups  had  identical  exposure  to  written 
instructions  and  actual  practice  time.  The  only  difference 
in  procedure  between  the  experimental  and  control  groups 
was  the  availability  of  supplemental  CAI  feedback  during 
practice  trials  one  to  sixteen  for  the  experimental  groups. 
Table  2 summarizes  the  treatment  given  to  each  group. 

MEASURES 

A variety  of  measures  were  taken  on  subjects'  performance 
on  the  apparatus,  including  clutch  depression  time,  time 
spent  in  each  gear,  RPM  and  speed  across  time  and  many  more. 
For  the  purpose  of  determining  the  relative  proficiency 
of  the  control  and  experimental  groups,  only  two  measures 
were  used. 

1.  Time-Off-Target : Actual  time  to  reach  55  MPH  - 
22  seconds  (target  time). 

2.  Smoothness:  The  mean  squared  deviation  from  the 
regression  line  calculated  for  that  trial. 

These  two  measures  were  utilized  because  they  were  defined 

as  criteria  for  good  performance  in  all  training  sessions. 

The  more  specific  measures  such  as  clutch  depression  time, 

time  in  each  gear,  etc.,  were  not  as  appropriate  for 

evaluating  systems  performance  since  they  were  only 

measures  of  subsystem  performance  and  were  not  defined  in 

terms  of  good  or  bad  systems  performance. 
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RESULTS 


No  significant  difference  was  found  in  the  time-off- 
target  measure  (see  Figures  4,  5,  and  6;  and  Tables  3 and  4). 
Smoothness  score  means  and  deviations  for  the  sum  of  the 
last  four  trials  are  presented  in  Figure  7 for  the  control 
and  experimental  groups. 

No  significant  differences  between  male  and  female 
control  groups  existed,  so  the  subjects  from  these  two  | 

groups  were  pooled.  Only  the  last  four  trials  were  used 
to  test  the  effectiveness  of  the  training  method,  because  i 

they  were  the  only  sessions  where  all  groups  received  | 

intrinsic  real-time  feedback  without  terminal  feedback.  j 

After  the  initial  six  trials,  MS  scores  only  showed  slight 
improvement  with  practice  for  both  experimental  and  control 
groups.  In  examination  of  the  control  group's  MS  scores. 

Figure  8 shows  that  high  quality  scores  were  as  likely  to 
occur  early  in  the  sequence  as  later,  and  the  occurrence 
of  a single  high  quality  trial  did  not  predict  an  increased 
likelihood  of  a high  score  on  the  immediately  following 
trial . 

A one-way  analysis  of  variance  (Table  5)  of  MS  scores 
indicated  a p value  of  .035.  Both  the  terminal  and  the 
graphic  feedback  conditions  produced  significantly  better 
scores  than  the  control  condition  but  did  not  differ 
significantly  between  each  other  (see  Table  6) . 
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Figure  4 

Control  and  Experimental  Groups  Time-off-Target 
Absolute  error  for  the  Sum  of  Trials  17-20. 
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Figure  S 

Control  and  Experimental  Groups  Time-off -Target 
Constant  Error  for  the  Sura  of  Trials  17-20. 
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Table  3 

One-Way  Analysia  of  Variance  on 
Time-off-tarMt  Absolute  Error 
Scores  for  Trials  17-20.* 


Source 

MS  F 

E 

Conditions 

3 

1492145.  1.25 

.295 

Within 

57 

1196961. 

TOTAL 

59 

*sum  of  each 
17  through  20 

subject's 

• 

Absolute  Error  scores 

for  trials 
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Table  4 

One-Way  Analysis  of  Variance  on 
Time-off- target  Constant  Error 
Scores  for  Trials  17-20.* 


Source 

MS 

F 

£ 

Conditions 

3 

1193776. 

.81 

.448 

Within 

57 

TOTAL 

59 

*sum  of  each  subject's  scores  for  trials  17  through  20. 


mean  score  # 


*1  standard  deviation 


-1  standard  deviation 
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Group  Group  Group 


Figure  7 

Control  and  Experimental  Groups  Smoothness  (MS)  Scores 
for  the  Sum  of  Trials  17-20. 
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Table  5 

One-Way  Analysis  of  Variance  on 
Smoothness  (MS)  Scores 
for  Trials  17-20.* 


Source 

MS 

F 

£ 

Conditions 

2 

66098.188 

4.70 

.013 

Within 

57 

14072.590 

TOTAL 

59 

*sum  of  each  subject’s  trials  17  through  20. 


[ 

i 

h 
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[ 

i Table  6 i 

i Duncan  Multiple  Range  Test  on 

' Smoothness  (MS)  Scores 

for  Trials  17-20.  , 

) 

I 

i 

Control  Graphic  Terminal  ■ 

Control  

Graphic  83.661*  i 

' 

Terminal  102.247*  18.586  


*probability  less  than  .05 


When  performance  during  the  training  trials  (1-16) 
are  examined  (Figure  9) , it  is  apparent  that  the  initial 
performance  improvement  in  MS  scores  took  place  in  the 
first  block  of  eight  trials  and  that  by  the  end  of  this 
block  the  two  experimental  groups  were  superior  to  the 
control  group.  It  is  important  to  note  that  during 
trials  1-8,  subjects  in  the  graphic  feedback  group  received 
real-time  graphic  feedback.  However,  on  the  second  eight 
trials  (9-16),  subjects  in  the  graphic  feedback  group 
received  only  intrinsic  real-time  feedback  which  resulted 
in  decrement  in  performance  that  was  not  compensated  for 
until  the  test  sessions  17-20.  An  analysis  of  variance 
run  on  data  for  trials  9-16  (Table  7)  showed  that  a 
significant  difference  existed  among  the  three  groups. 

The  Duncan  Multiple  Range  Test  (Table  8)  indicated  that 
this  difference  was  not  between  the  control  and  the  graphic 
feedback  or  the  graphic  and  terminal  feedback  groups,  but 
between  control  and  terminal  feedback  groups  (see  Figure  10) , 
the  terminal  feedback  scores  being  significantly  superior 
(p  < .05)  between  trials  9-16. 
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Figure  9 

Control  and  Experimental  Groups  Smoothness  (MS)  Scores  for  Trials 


Table  7 

One-Way  Analysis  of  Variance  on 
Smoothness  (MS)  Scores 


for  Trials  9-16. * 

Source 

if 

F 

E 

Conditions 

2 

235797.000 

7.01 

.002 

Within 

S7 

33646,473 

TOTAL 

S9 

•sum  of  each  subject's  trials  9 through  16. 
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Table  8 

Duncan  Multiple  Range  Test  on 
Smoothness  (MS)  Scores 
for  Trials  9-16. 

Control  Graphic  Terminal 

Control  

Graphic  107.337  

Terminal  213.764*  106.427  

*probability  less  than  .05 


i 
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DISCUSSION 


The  lack  of  any  experimental  differentiation  between 
Terminal  Numerical  feedback  and  Graphic  Analytic  feedback 
was  most  probably  a result  of  the  relative  simplicity  of 
the  desired  psychomotor  output.  Both  types  of  feedback 
served  equally  well  to  fix  good  performance  examples  in 
the  students'  behavior  repertory,  and  to  inhibit  the 
repetition  of  poor  performance  examples.  The  variability 
of  performance  was  strikingly  reduced  under  both  experimental 
conditions,  as  indicated  by  a reduction  in  the  smoothness 
score  standard  deviations  to  30  percent  of  the  control  group 
values.  This  sharp  reduction  in  response  variability  also 
indicated  that  some  genuine  enhancement  of  psychomotor 
skills  took  place  under  the  experimental  conditions.  It 
was  anticipated  that  Graphic  feedback  would  be  superior 
to  the  simpler,  non- concurrent  Numerical  feedback,  but 
this  was  not  the  case.  In  this  relatively  simple  task, 
the  numerical  feedback  alone  seems  to  have  been  sufficient 
for  a subject  to  evaluate  the  effectiveness  of  various 
performance  tactics,  without  needing  to  see  a graphic 
display  of  performance  relative  to  an  optimum  performance 
track.  The  fact  that  subjects  in  the  Graphic  feedback 
group  did  not  perform  better  than  the  Terminal  feedback 
group,  even  during  the  first  eight  trials,  suggests  that 
this  is  true.  To  be  effective  in  shaping  new  performance 
skills,  a command  display  must  be  superior  to  the  performance 
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on  the  apparatus  without  the  real-time  aid.  Otherwise, 
there  is  no  point  in  adding  artificial  feedback,  since 
subjects  must  eventually  depend  completely  on  intrinsic 
feedback. 

It  may  be  that  with  tasks  requiring  more  precision  of 
motor  output,  the  analytic  display  format  would  have  some 
training  advantage  over  simpler  terminal  feedback  of 
numerical  scores.  The  subjects  in  this  study,  in  all 
groups,  exhibited  considerable  flexibility  in  performance 
output.  Few  of  them  demonstrated  the  rigidity  of  an 
inadequate  motor  output  that  often  characterizes  poor 
psychomotor  performance.  An  evaluation  of  individual 
subject  records  indicated  that  both  feedback  formats  were 
of  some  use  in  removing  bad  performance  habits  that  might 
otherwise  be  intractable  to  most  training  procedures. 

It  is  felt  that  in  both  cost  and  effectiveness,  a CAI 
system  of  psychomotor  instruction  could  be  a match  for  many 
one-to-one  instruction  procedures  involving  close  supervision, 
and  would  be  superior  to  almost  any  classroom  environment 
with  unsupervised  practice. 
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APPENDIX  A 
Operations  Manual 


OPERATIONS  MANUAL 


The  driving  simulator  that  you  will  be  working  with  In  this  study 
operates  much  like  a normal  4 speed  manually  operated  car  or  truck.  Your 
task  will  be  to  learn  how  to  control  the  simulator  In  such  a way  that 
road  speed  (as  Indicated  by  a speedometer)  Increases  gradually  with  as 
little  difference  as  possible  In  the  rate  of  speed  Increase  In  different 
gears.  At  the  same  time  you  should  attempt  to  have  as  small  a drop  In 
speed  as  possible  when  you  shift  from  one  gear  to  another.  The  simulator 
responds  very  much  like  a car  or  truck  moving  up  a gradually  sloping 
hill.  For  this  reason  the  speed  will  tend  to  drop  quite  quickly  If  the 
clutch  Is  left  disengaged  for  very  long  periods  of  time. 


Performance  Requirements 

You  win  be  required  to  shift  through  the  gears~lst  to  2nd  to  3rd  to 

4th— while  trying  to  accomplish  two  objectives. 

1.  Increase  the  speed  slowly  and  steadily  from  0 to  55  mph  with 

as  little  fluctuation  In  this  steady  Increase  In  speed  as  possible. 
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2.  This  steady  Increase  should  be  planned  so  that  the  time  Interval 
between  0 mph  to  55  mph  Is  as  close  to  22  seconds  as  possible. 

Reaching  55  mph  before  or  after  22  seconds  will  not  be  considered  perfect 

performance.  However,  If  you  make  It  to  55  mph  Just  at  22  seconds,  but 

do  not  maintain  a steady  Increase  In  speed  you  also  will  be  considered  to 

have  fallen  short  of  perfect  performance.  The  Ideal  performance  on  the 

task  then  Is  one  where  a gradual  steady  Increase  In  speed  will  result  In 

obtaining  55  mph  Just  when  22  seconds  has  elapsed.  Unless  the  speed  Increase 

Is  steady,  hitting  55  mph  right  on  time  will  be  of  small  value.  If  the  Ideal 

performance  Is  plotted  on  a graph.  It  looks  much  like  that  shown  In  Figure  1. 


Figure  1 

Ideal  Performance 


i 


The  speed  Is  marked  on  the  loft  side  from  0 to  60  mph  and  the  time  Is 
marked  across  the  bottom.  The  line  moving  from  the  zero  point  to  the 
right  Is  a measure  of  the  speed  that  you  should  be  traveling  at  any  time 
during  a shifting  session.  Be  sure  to  note  that  the  Increase  in  speed 
Is  gradual  and  that  at  no  time  does  the  rate  of  Increase  change. 
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Instrmwnts  and  Controls 

A speedometer  (indicating  road  speed)  and  a tachometer  (indicating 
engine  speed)  will  be  displayed  on  a video  screen  directly  in  front  of 
the  seat  in  the  simulator. 


The  scales  displayed  on  the  video  screen  indicate  speed  and  RPM  scales. 
They  are  controlled  by  an  accelerator,  a clutch  and  a 4 speed  transmission. 
The  clutch  and  accelerator  positioning  is  shown  below. 


clutch  accelerator 

pedal  pedal 
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The  shift  pattern  Is  the  standard  4 speed  shift  pattern. 


Reverse 


Reverse  and  Low  will  not 
be  used  for  this  study, 
only  1st,  2nd,  3rd,  and 
4th  will  be  used. 


In  order  to  shift  from  one  gear  to  another  In  a manual  transmission, 
the  clutch  pedal  must  be  depressed,  the  shift  must  be  made  from  one  gear 
to  the  next  and  then  the  clutch  pedal  must  be  released.  The  proper 
sequence  of  events  Is  then  as  follows: 


Repeat  the  Same 
Process  when 
Shifting  from 
2nd  to  3rd  and 
from  3rd  to  4th 
gear 


How  to  Achieve  a Steady  Increase  In  Speed 

1.  Make  the  Increase  In  speed  within  each  gear  as  steady  as  possible. 
Also  make  the  rate  of  speed  Increase  In  all  gears  as  close  to  the 
same  as  possible. 
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MILES  PER  HOUR 


2.  Make  the  shifting  time  as  small  as  possible. 
Study  Figure  2 closely. 

r 


Figure  2 

Correct  Performance 


3.  Shift  to  the  next  higher  gear  as  the  engine  RPM  approaches  2300. 
The  RPM  should  not  be  stabilized  at  2300  when  the  shift  takes 
place,  but  Instead  should  be  approaching  2300  as  the  shift  to  the 
next  higher  gear  takes  place. 

4.  The  proper  starting  RPM  for  the  next  gear  should  be  registered 
when  the  clutch  Is  engaged  (See  Table  1).  This  RPM  should  be 


40 


MOTOR  REVOLUTIONS  PER  MINUTE 


6 


Figure  3 below  shows  the  Increase  In  RPM  In  the  same  way  that  earlier 
tables  showed  Increase  In  speed.  Note  that  the  RPM  reading  does  not 
stabilize  at  a specific  RPM  before  you  shift.  The  RPM  should  be  Increasing 
until  the  shift  takes  place.  Also  the  RPM  should  begin  to  Increase 
again  right  when  the  clutch  Is  engaged. 


Figure. 3 

CORRECT  RPM  CONTROL 

I 
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Training  Procedure 


1.  While  sitting  In  the  simulator,  experiment  with  the  gears  so  you 
are  familiar  with  their  location. 

2.  You  will  then  be  Instructed,  via  the  video  screen,  to  depress  the 
clutch,  shift  Into  first  gear  and  allow  the  motor  to  Idle. 

3.  When  you  hear  a beeping  sound  you  are  to  shift  through  the  gears 
from  1st  to  4th  as  smoothly  as  possible,  attempting  to  reach 

55  mph  22  seconds  after  the  first  beep  sounds.  After  22  seconds, 
another  beep  will  sound.  If  you  reach  55  mph  before  the  22  seconds 
elapse,  you  should  plan  on  slowing  your  rate  of  acceleration  on  the 
next  trial.  If  you  do  not  reach  55  mph  before  the  second  beeping 
sound,  don't  stop,  continue  to  steadily  Increase  your  speed  until 
you  do  reach  55  mph.  Your  score  Is  dependent  on  steadiness  of 
acceleration  as  much  as  It  Is  dependent  on  how  close  you  get  to  the 
22  second  target  time.  If  you  do  overshoot  the  buzzer,  you  should 
plan  on  speeding  up  your  rate  of  acceleration  on  the  next  trial. 

4.  Upon  completing  the  series  of  shifts,  you  will  be  instructed  to 
shift  Into  neutral  and  take  your  foot  off  the  accelerator. 

5.  The  shifting  sequence  will  then  be  repeated. 


Operations  Manual  Exam 


1.  When  the  target  time  designated  to  reach  55  mph  is  reached  it  will 
be  signaled  by: 

a.  a flashing  light. 

b.  a shutdown  of  the  display. 

c.  a beeping  sound. 

d.  the  word  TIME  printed  across  the  display. 

e.  it  will  not  be  signaled. 

2.  It  is  recoRinended  that  when  shifting  out  of  1st,  2nd,  and  3rd  gears 
that  the  rpm: 

a.  have  a stable  reading  of  1750  rpm. 

b.  have  a stable  reading  of  1250  rpm. 

c.  have  a stable  reading  of  2300  rpm. 

d.  be  moving  toward  1750  rpm. 

e.  be  moving  toward  2300  rpm. 

3.  A11  of  the  actions  listed  below  will  help  in  achieving  good  performance 
on  the  shifting  task  except  one.  Which  alternative  will  help  the  least 
in  obtaining  good  systems  performance? 

a.  keep  the  speed  increase  within  each  gear  as  steady  as  possible. 

b.  accelerate  quickly  and  smoothly  in  each  gear. 

c.  keep  the  shift  time  as  small  as  possible. 

d.  keep  the  rate  of  speed  increase  in  each  gear  the  same. 

e.  be  accelerating  in  each  gear  up  until  the  time  the  shift  takes  place. 

4.  Upon  engaging  the  clutch  in  2nd  or  3rd  gear,  the  rpms  should  be  at: 

a.  1250  rpm  and  climbing. 

b.  2300  rpm  and  climbing. 

c.  1750  rpm  and  climbing. 

d.  a steady  1250  rpm. 

e.  a steady  1750  rpm. 

5.  Upon  engaging  the  clutch  in  4th  gear  the  rpm  reading  should  be  at: 

a.  1250  rpm  and  climbing. 

b.  2300  rpm  and  climbing. 

c.  1750  rpm  and  climbing. 

d.  a steady  1250  rpm. 

e.  a steady  1750. 

6.  The  ideal  time  required  to  get  from  0 mph  to  55  mph  for  this  test  is: 


a. 

18 

seconds 

b. 

20 

seconds. 

c. 

22 

seconds 

d. 

24 

seconds 

e. 

26 

seconds 

L 
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If  you  do  not  reach  55  mph  In  the  desired  time  period  you  should: 

a.  stop  acceleration  at  once. 

b.  continue  to  accelerate  smoothly  until  you  reach  55  mph. 

c.  Just  push  the  accelerator  pedal  to  the  floor. 

d.  hold  the  speed  steady  at  the  speed  you  obtained  on  the  trial 
before  passing  the  desired  time  period. 

e.  there  Is  nothing  In  particular  that  you  should  do. 
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APPENDIX  B 


Apparatus  Hardware  and  Sensors 


s 


Schematic  1 is  representative  of  the  RPM  and  speed 
sensors.  The  intermittent  light  causes  resistance  changes 
in  the  photo-diode,  which  produces  a fluctuating  voltage 
at  the  top  of  the  Rl.  This  signal  is  capacitor  coupled 
by  Cl  to  remove  the  offset  from  (p,  and  D1  clamps  negative 
spikes.  T1  provides  current  sinking  capability  to  switch 
the  Schmidt  Trigger  (7413),  which  produces  a TTL  square 
wave.  The  7402,  R4 , T2 , and  D2  provide  a standard  cable 
driving  circuit  to  enter  the  12  bit  I/O. 

Schematic  2 shows  the  circuit  for  sensing  the  shift 
position.  The  reed  switches  mounted  around  the  gear  shift 
cause  the  input  to  El  to  go  to  ground  when  they  are  closed. 
Rl  holds  the  input  high  when  the  switch  is  open.  To  read 
the  shift  position,  a pulse  is  transmitted  from  the  12  bit 
I/O.  This  pulse  is  received  and  inverted  by  the  R4,  D2, 

D3,  E3,  E2  circuit,  and  is  used  to  strobe  the  six  gates 
monitoring  the  reed  switches.  R2,  T1 , and  D1  complete  the 
driving  circuit  which  connects  to  the  digital  I/O.  E4 , 

LED,  and  R3  provide  a monitoring  circuit  to  use  in  adjusting 
the  positions  of  the  reed  switches. 
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RPM  and  Speed  Sensors 


7417  LEO 


Shift  Position  Sensors 


APPENDIX  C 

Hierarchical  Instruction  Set 


A 


Hierarchical  Instruction  Set 


The  Huaan  Factors  Laboratory  has  done  some  pilot  work 
in  developing  a CAI  program  that  utilizes  a hierarchical 
instruction  format  consisting  of  summary  data,  discrete 
error  messages  and  real-time  command  modeling.  An  example 
of  this  instruction  format  is  shown  in  the  following  flowcharts. 
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CAI  System  Implementation  Guidelines 

It  is  a complex  process  to  program  a specific  inventory 
of  interactions  between  the  computer,  the  student,  and  the 
machinery  the  student  is  learning  to  control.  We  identified 
six  areas  in  which  formal  decisions  have  to  be  made  concerning 
the  details  of  such  interactions. 

1.  Selection  of  information  to  be  monitored  by  the 
computer.  The  application  of  computer  feedback 
techniques  to  psychomotor  training  requires  a 
variety  of  sensors,  or  "instrumentation," 
attached  to  the  device  whose  operation  is  being 
learned.  This  instrumentation  communicates  two 
different  types  of  data  to  the  computer: 

a.  Information  referring  to  operator's  motor 
output.  Sensors  on  the  controls  operated 
by  the  student  can  detect  the  position  of 
the  controls,  the  pressures  and  torques 
exerted  on  the  controls,  and  the  rate 
and  range  of  change  in  these  values. 

By  analyzing  these  data,  it  is  possible 
to  determine  the  actual  manipulation  of 
the  controls  by  the  student-operator. 

b.  Information  referring  to  the  dynamic  condition 
of  the  mechanical  system  being  operated. 

Sensors  on  various  operating  elements  of 

the  system  can  detect  such  values  as  RPM  of 
rotating  components,  internal  pressures  in 
hydraulic  components,  and  other  kinetic 
forces  indicating  the  actual  internal 
operating  conditions  of  mechanical  devices. 

Either  or  both  of  these  sets  of  data  can  be  used 
to  provide  feedback  concerning  his  performance 
to  the  trainee. 

2.  Selection  of  information  to  be  presented  to  the 
student  operator.  Information  may  be  fed  back 
by  computer  as  either  a visual  display  on  a TV 
type  screen  or  as  some  form  of  acoustic  display. 

The  contents  of  any  of  these  displays  may  be 
generally  characterized  as  being  one  or  another 
of  these  three  types: 
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a.  Numerical  scores,  which  may  represent  a 
value  of  a single  dimension  of  performance 
quality,  or  may  be  weighted  composite  indices 
of  several  dimensions  of  performance  quality. 

b.  Analytic  displays,  which  may  combine  one  or 
more  performance  variables  in  a graphic 
display  against  time,  or  as  a function  of 
other  variables. 

c.  Error  messages,  which  contain  English  language 
statements  describing  deviations  of  control 
outputs  or  system  conditions  from  the  nominal 
ideal  model. 

Although  displays  of  extraordinary  complexity 
can  be  arranged,  the  most  useful  ones  are  generally 
those  plotting  some  single  motor  output  function 
or  system  condition  against  time. 

3.  Selection  of  the  rate  and  timing  of  feedback 

to  the  student.  If  tasks  are  relatively  brief, 
a single  display  may  be  presented  once  at  each 
completion  of  a discrete  event.  With  tasks  of 
substantial  duration,  the  feedback  may  include 
static  displays  presented  at  discrete  intervals, 
or  as  a continuously  changing  display.  In  either 
case,  the  best  rate  for  updating  displays  had  to 
be  experimentally  determined. 

Any  display  occurring  during  the  performance  of 
a psychomotor  task  constitutes  an  additional  work- 
load, competing  for  attention  with  other  task 
elements  and  profoundly  changing  the  nature  of 
the  task  being  performed.  The  presence  of  a 
continuously  updated  display  may  add  elements  of 
a tracking  task  to  a performance  situation  that 
otherwise  would  not  include  a tracking  requirement. 

4.  Selection  of  the  complexity  and  style  of  messages, 
scores  and  displays.  In  the  case  of  error 
messages,  the  complexity  of  vocabulary  and  grammar 
must  be  adjusted  to  the  educational  level  of  the 
individuals  being  instructed.  Numerical  scores 
must  be  scaled  to  an  optimal  number  of  discrete 
steps  and  significant  digits. 

5.  Provision  may  be  made  for  storage  of  any  of  the 
parameters  generated  during  the  instruction  process. 
This  would  include  records  of  operator  output, 
system  condition  and  also  the  array  of  feedback 
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displays  and  scores  actually  presented  to  the 
student.  If  such  data  are  stored  over  a long 
series  of  trials,  the  student  may  be  offered 
supplemental  feedback  showing  the  rate  and 
direction  of  progress  made  over  the  series. 
Information  can  also  be  presented  to  the  student 
in  terms  of  his  relationship  to  larger  populations, 
such  as  centile  or  Z-scores. 

6.  Some  provision  should  also  be  included  for  testing 
the  transfer  of  the  student's  newly  acquired 
performance  capabilities  from  the  feedback  to 
non-feedback  conditions. 
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Discussion  of  the  Present  Study  in  Terms  of 
Operant  Conditioning  Theory 


Computer-assisted  instruction  represents  an  attempt 
to  organize  the  process  of  teaching  in  terms  of  what  is 
known  about  human  learning.  One  set  of  basic  principles 
which  are  relevant  to  this  process  are  those  of  B.  F.  Skinner, 
which  emphasize  the  importance  of  immediate  reinforcement, 
gradual  progression  or  shaping  of  responses,  the  maximization 
of  the  probability  of  success,  and  the  minimization  of 
behavioral  inhibition  due  to  punishment.  While  none  of 
our  training  procedures  were  incompatible  with  these 
principles,  there  was  a substantial  divergence  of  emphasis 
between  what  we  did  aid  what  might  be  expected  in  terms 
of  a traditional,  operant  conditioning  oriented  training 
program. 

a.  Learning  to  do  the  job  correctly  was  presumed 
to  be  its  own  reward.  No  attempt  was  made  to 
introduce  the  continuous  stream  of  praise  or 
other  secondary  reinforcements  which  are  often 
associated  with  cognitive  CAI . 

b.  The  student  was  presumed  to  be  sufficiently  alert 
and  competent  to  shape  his  own  motor  output  to 
closer  and  closer  approximations  of  the  ideal 
performance  model  if  the  model,  his  own  output, 
and  the  points  of  difference  between  them 

were  numerically  or  graphically  displayed.  No 
attempt  was  made  to  establish  a reinforcement 
pattern  for  progressive  improvement  in  performance 
other  than  the  knowledge  of  results. 

c.  Where  extraordinarily  complex  performance  is 
called  for;  such  as  genuinely  independent 
movement  of  two  limbs,  or  coordinated  movements 
in  several  dimensions,  a more  formal  "shaping" 
procedure  may  be  desirable.  This  may  also  be 
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desirable  in  cases  where  the  activity  to  be 
performed  is  quite  different  from  any  motor 
output  patterns  likely  to  pre-exist  in  the 
average  student's  behavior  repertory.  Shaping 
procedures  may  easily  be  programmed  into  a 
CAI  psychomotor  training  system.  In  terms  of 
our  present  system,  the  addition  of  a shaping 
option  would  call  for  creating  and  storing  a 
series  of  ideal  performance  models  with  progressive- 
ly stricter  demands  on  accuracy  of  timing  and 
control,  and  changing  the  reference  model  used 
as  the  students  become  more  proficient. 

d.  The  presentation  of  error  messages  has  some  of 

the  characteristics  of  a punishment,  or  an  aversive 
conditioning  process.  This  may  frustrate  or 
inhibit  poor  students,  and  may  tend  to  depress 
the  flexibility  of  their  behavioral  responses. 

This  danger  should  be  minimized  by  programming 
certain  constraints  into  the  system  of  error- 
message  presentation.  The  messages  should  be 
organized  into  a hierarchy  of  importance,  and 
only  one  or  two  presented  to  a student  in  any 
given  trial.  In  addition,  each  error  message 
should  be  followed  by  an  opportunity  to  practice 
and  correct  the  error  separately  from  the 
integrated  performance  pattern  being  taught, 
if  at  all  possible,  and  the  correction  should 
be  immediately  reinforced  with  some  type  of 
praise,  even  if  the  integrated  performance  score 
does  not  show  any  immediate  improvement. 


' Reference:  Skinner,  B.  F.,  The  Technology  of  Teaching. 

; Appleton-Century-Crofts : New  York,  1968 . 
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APPENDIX  F 


Description  of  the  Computer  System 
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Description  of  the  Computer  System 


The  following  is  a list  of  the  parts  of  the  Human  Factors 
Laboratory  computer  system  that  are  utilized  in  the  operation 
on  the  simulator. 


Following  this  is  a listing  of  a minimum  system  necessary 
to  operate  the  simulator  providing  some  alterations  in  the 
software  are  made. 


Current  Components  Utilized  in  the  Current  System 


From  Digital  Equipment  Company: 


Price* 


PDP8E-AE  Rack  Mountable  8K  system:  $ 5,650 

Computer,  8K  Core  Memory  and  Teletype 
Control  (rack  mountable,  slides  included). 

Unit  consists  of: 

1 KK8-E  Central  Processor 
1 MC8-EJ  8K  Core  Memory  and  Memory 
Extention  Control 
1 KC8-EA  Programmer's  Console 
1 KL8-E  Console  Teletype  Control 
1 Combination  Power  Supply,  Chassis 
and  OMNIBUS  with  20  Quad  Bus 
Slots 


MR8-EC  256  Word  OS/8  ROM  for  TD8-E  systems  800 

DK8-EP  Real-time  Clock,  Programmable  810 

VT8-EA  High-speed  Alphanumeric  and  Graphics  2,050 

Video  Display  Terminal  and  Control 
(with  64  character  line  buffer,  upper 
case  5x7  dot  matrix  60  Hz) . 

TD8-EM  OMNIBUS  DECtape  Control  and  Dual  DECtape  5,000 
Drive.  Data  transfer  via  programmed  I/O. 

Max  4 TD8-E  Controls  per  system  (rack 
mountable) . 

AH04  Sample  and  Hold  Option  300 

ADOl-AP  10-Bit  Analog-to-Digital  Conversion  System  2,400 
up  to  32  channels.  Gains  are  programmable 
up  to  +10  volts  (channels  are  implemented 
with  the  A124  option) 

KE8-E  Extended  Arithmetic  Element  (EAE)  1,080 
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Non-Digital  Equipment  Company  Equipment: 

Price' 

LP8  Tally  Line  Printer  and  Controller 
(200  lines  per  minute) 

$ 5,500 

24  K of 

Plessy  Core 

4,500 

TOTAL  COST  OF  EQUIPMENT  USED 

$33,635 

Minimum  System  Required  to  Operate  Simulator  (with  s 

oftware 

alterations) 

PDP8E-AE 

Base  System 

$ 5,650 

VT8-EA 

Video  Display 

2,050 

KA8-E 

Extender  Interface  for  Positive 

I/O  Device 

250 

DR8-EA 

12  Bit  Buffered  I/O 

500 

DK8-EP 

Real-Time  Clock,  Programmable  Clock 

810 

ADOl-AD 

10-Bit  Analog-to-Digital  Conversion 

2,400 

LT33-DC 

ASR33  Synchronous  Read  and  Punch, 
Friction  Feed 

1,620 

LP8 

Tally  Line  Printer  and  Controller 

5,500 

TOTAL  COST  OF  MINIMUM 
OPERATING  EQUIPMENT 

$18,780 

* Prices  are  the  costs  at  the  time  of  purchase  (approximately 
1973)  . 


65 


APPENDIX  G 


Sample  of  Computer  Printout  Showing  Storage  Format 
and  Graphic  Representation  of  Data  Collected 
During  a Single  22  Second  Trial 
Involving  Three  Shift  Points 
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Target  » Actual  time  to  reach  55  mph  - 22  seconds. 

terminated  after  28  seconds  and  the  time  required  to  get  to  55  mph  Is  estimated. 


